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ABSTRACT 

The  pneumococcal  vaccine  is  underutilized  according  to  self-reported  surveys  and  the  full 
scope  of  the  problem  remains  undetermined.  Objective:  Primary  aims  were  to  determine  if 
the  documentation  of  pneumococcal  vaccination  varied  between  clinics  for  patients  65  and 
older;  and  to  compare  the  results  with  national  survey  reports  and  claims  data.  Methods:  368 
medical  records  were  examined  in  this  prevalence  survey  with  268  records  from  the  internal 
medicine  clinic  (IMC)  and  100  records  from  the  family  practice  clinic  (FPC)  at  the  Wilford 
Hall  U.S.  Air  Force  Medical  Center.  Results:  The  documentation  rates  were  92.9%  and 
69.0%,  for  IMC  and  FPC  respectively.  This  difference  in  documentation  rates  was 
statistically  significant  (chi-square  =  35.462;  df  =  1;  p  <  0.001)  and  both  clinics  exceeded  the 
rates  found  in  surveys  and  claims.  Conclusions:  Clinics  differ  in  their  vaccine  documentation 
and  self-  reported  national  survey  data  might  not  accurately  reflect  the  prevalence  of 
pneumococcal  vaccination. 
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INTRODUCTION 

Streptococcus  pneumoniae  (pneumococcus)  is  a  pathogenic  bacterium  that  affects 
children  and  adults.  It  is  a  leading  cause  of  illness  and  death  among  the  elderly  and  in  persons 
with  certain  medical  problems.1  As  the  U.S.  population  continues  to  age  and  incur  increased 
medical  expenditures,  it  is  becoming  vital  that  we  minimize  preventable  diseases.  The 
pneumococcal  vaccine  in  persons  65  and  older  is  currently  markedly  underutilized  according 
to  several  recent  surveys  and  claims  data. 

The  National  Health  Interview  (NHIS)  is  an  ongoing  annual  cross-sectional  survey  of 
households  that  is  given  annually  to  a  nationwide  representative  sample  of  civilians.  The 
1995  NHIS  data  indicated  that  for  persons  65  and  older  34%  responded  that  they  had  ever 
had  received  the  pneumococcal  vaccine.2  The  Behavioral  Risk  Factor  Surveillance  System 
(BRFSS)  is  an  ongoing  random  digit-dialed  telephone  survey  of  U.S.  civilian  persons  18 
years  of  age  and  older.  The  BRFSS  data  for  persons  65  and  older  for  ever  having  had  a 
pneumococcal  vaccination  for  years  1995,  1997,  and  1999  was  38.4%,  45.8%,  and  54.9% 
respectively.3  The  Medicare  Current  Beneficiary  Survey  (MCBS)  is  a  continuous 
longitudinal  survey  of  Medicare  beneficiaries.  The  1996  MCBS  revealed  that  45.1%  of 
eligibles  65  and  older  reported  ever  having  received  a  pneumococcal  vaccination.4  Analysis 
of  Medicare  claims  by  the  Health  Care  and  Finance  Administration  for  the  period  1991-1998 
trended  with  the  surveys  and  showed  that  only  31%  of  eligible  beneficiaries  recorded  as 
having  received  pneumococcal  vaccination  despite  the  fact  that  pneumococcal  vaccination 
has  been  a  covered  benefit  since  1981 ,5  There  were  significant  differences  between  ethnic 
groups,  socioeconomic  classes,  and  states.  In  general  however,  these  statistics  are  quite 
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alarming  given  the  mortality,  morbidity,  and  health  care  costs  associated  with  this  vaccine 
preventable  disease.  If  accurate,  these  numbers  do  not  bode  well  for  the  nation’s  healthcare 
budget  given  our  aging  baby  boomer  population. 

The  full  scope  of  the  problem  of  low  pneumococcal  vaccination  rates  remains  to  be 
determined.  Attempts  have  been  made  to  assess  the  validity  of  self-reports  elicited  in  surveys 
by  comparing  the  information  in  the  medical  records  to  that  provided  during  interviews  and 
questionnaires.  The  two  sets  of  records  often  disagreed.6  A  study  looking  at  clinicians’ 
records  found  that  clinicians  do  not  document  immunizations  for  most  of  their  adult  patients.7 
An  influenza  and  pneumococcal  vaccination  self-report  study  found  pneumococcal 

•  o 

vaccination  reporting  was  less  valid,  particularly  for  more  distant  vaccination. 

To  elucidate  the  problem  of  vaccination  rates  in  persons  65  and  older  to  S. 
pneumoniae  a  study  involving  a  medical  record  review  for  documentation  of  pneumococcal 
vaccine  was  desirable.  A  sample  from  two  different  clinic  types  was  chosen  for  medical 
record  review  to  see  how  they  compare  to  each  other  in  regards  to  documenting 
pneumococcal  vaccination  and  to  the  self-reported  rates  from  surveys  and  claims  data.  The 
results  of  the  study  may  help  clarify  the  extent  of  pneumococcal  vaccination  and  shed  light 
on  those  factors  which  lead  to  increased  or  decreased  vaccination  rates  in  persons  65  years  of 
age  or  older. 
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METHODS 

Study  Design 

A  prevalence  survey  was  used  to  compare  the  documentation  of  pneumococcal 
vaccination  in  the  medical  records  of  persons  65  and  older  in  the  IMC  and  FPC. 

The  computer  program,  nQuery  Advisor,  Version  4.0  was  used  to  determine  the 
minimum  sample  size.9  The  study  was  designed  to  detect  a  difference  in  reporting  accuracy 
of  20%,  with  a  two-sided  alpha  of  0.050  and  a  power  of  70%.  The  predicted  reporting 
accuracy  was  70%  for  the  IMC  and  50%  for  FPC.  Based  on  these  inputs,  the  minimum 
sample  size  required  was  155. 

Review  and  approval  of  the  study  proposal  was  done  at  Wilford  Hall  Medical  Center, 
Lackland  Air  Force  Base  and  the  University  of  Texas  Health  Science  Center  at  Houston. 

Data  Collection 

Charts  were  reviewed  solely  by  the  principal  investigator  to  determine  if  there  was 
documentation  of  pneumococcal  vaccination  in  the  medical  record.  No  unique  identifiers 
were  abstracted  from  the  medical  records;  confidentiality  and  record  security  was  maintained 
at  all  times.  The  medical  record’s  jacket,  problem  list,  vaccination  sheets,  or  documentation 
in  the  record  in  any  of  the  last  three  primary  care  visits  were  used  to  make  the  pneumococcal 
vaccine  documentation  determination.  If  there  was  any  doubt,  the  medical  record  was 
considered  as  not  having  documentation  of  pneumococcal  vaccination.  Due  to  time 
constraints  and  shortage  of  staff  to  pull  medical  records,  cluster  convenience  samples  of 
records  were  used  from  both  clinics.  The  records  were  evaluated  without  provider  knowledge 
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of  the  prevalence  survey.  The  entire  data  collection  period  spanned  2  weeks  and  covered  four 
distinct  collection  days. 

Charts  were  pulled  from  the  medical  records  area  and  taken  to  the  particular  clinic  in 
the  morning  or  afternoon  prior  to  a  patient’s  scheduled  visit,  the  provider  utilized  the  record 
and  made  documentation  of  the  encounter,  the  records  were  then  collected  and  taken  to  the 
coding  department,  and  finally  returned  to  the  medical  records  area.  I  screened  all  records 
going  through  the  coding  department  that  came  from  the  IMC  or  FPC  during  the  four  survey 
days  and  included  all  medical  records  in  the  prevalence  survey  that  indicated  an  age  of  65  or 
older  on  the  day  of  the  patient’s  visit. 

Data  Analysis 

The  data  were  entered  into  Minitab  software  to  facilitate  statistical  analysis  and 
presentation  of  data.  The  data  were  organized  into  a  two-by-two  (r  x  c)  contingency  table  and 
a  chi-square  test  was  used  to  test  for  statistical  significance  of  an  association  betw  een  the 
type  of  clinic  and  pneumococcal  vaccine  documentation.  Odds  ratios  and  confidence 
intervals  were  calculated  to  further  define  the  extent  of  any  association. 

RESULTS 

The  total  number  of  medical  records  reviewed  was  368  with  268  from  the  IMC  and 
100  from  the  FPC.  The  pneumococcal  vaccine  documentation  rates  were  92.9%  and  69.0%, 
for  IMC  and  FPC  respectively;  see  Table  1 .  This  difference  in  documentation  rates  was 
statistically  significant  (chi-square  =  35.462;  df =  1;  P  <  0  .001).  Furthermore,  an  association 
of  moderate  degree  was  found  to  be  present  between  clinic  type  and  documentation  of 
pneumococcal  vaccination.  The  data  suggest  that  the  odds  of  having  a  pneumococcal 
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vaccination  documented  if  one  was  seen  in  IMC  was  5.89  times  higher  than  if  one  was  seen 
in  FPC  with  a  narrow  confidence  interval  (3.13,  11 .07). 

The  patient  populations  that  generated  these  results  were  expected  to  be  similar  The 
clinics  were  anticipated  to  have  comparable  age  profiles  as  this  study  was  concerned  with  the 
65  and  older  population.  Table  2  showed  the  age  differences  between  the  two  clinics.  The 
IMC  had  ages  ranging  from  65  to  93  with  a  mean  of  73.4  years,  while  the  FPC  had  ages 
ranging  from  65  to  85  with  a  mean  of  67.0  years.  Table  3  revealed  that  87.0%  of  the  medical 
records  from  FPC  were  aged  65-69.  While  this  grouping  was  the  largest  for  the  IMC,  as  well, 
it  only  had  33.2%.  The  gender  breakdown,  as  shown  in  Table  4,  had  IMC  with  49.3%  males 
and  50.7%  females  and  the  FPC  with  43.0%  males  and  57.0%  females.  The  grades  for  the 
IMC  ranged  from  E-4  to  0-8.  The  grades  for  the  FPC  ranged  from  E-5  to  0-7.  The  grade 
frequencies  were  similar  in  the  two  clinics  with  the  most  common  grades  being  E7-E9  in 
both;  see  Table  5.  There  was  no  statistically  significant  difference  in  the  rates  at  which 
officers  and  enlisted  personnel  had  pneumococcal  vaccination  documented  in  their  records 
according  to  analysis  done  in  Tables  6  and  7.  Race  of  individuals  was  not  available  in  the 


medical  records. 


8 


DISCUSSION 

The  first  objective  of  this  study  was  to  determine  if  the  two  different  clinics  differed 
in  their  rates  of  pneumococcal  vaccination  documentation  in  the  medical  record.  A 
significant  difference  was  detected  in  the  documentation  of  pneumococcal  vaccination  in  the 
medical  records  between  the  IMC  and  the  FPC.  The  figure  of  92.9%  was  much  higher  than 
the  70%  predicted  pre-study  for  the  IMC.  The  69%  figure  for  the  FPC  also  much  better  than 
the  50%  predicted  pre-study.  The  finding  that  the  overall  documentation  for  both  clinics  was 
better  than  expected  is  encouraging  for  this  particular  facility.  However,  it  sheds  further  light 
on  the  variability  between  self-reported  vaccination  rates  and  medical  documentation.  This 
leads  to  the  second  objective  of  the  study  that  was  to  determine  how  these  documented  rates 
compared  with  survey  and  claims  data.  Both  clinics  far  exceeded  the  pneumococcal 
vaccination  rates  found  in  national  surveys  and  claims  data;  see  Figure  1  for  comparisons. 

A  reason  for  the  difference  between  the  two  clinics  could  be  that  they  were  clustered 
convenience  samples.  The  records  reviewed  for  the  survey  were  records  passing  through 
coding,  on  four  different  days,  on  their  way  back  to  the  medical  records  area.  Neither  the 
patients  nor  providers  were  aware  of  the  study.  The  facility  was  not  able  to  provide 
manpower  to  assist  with  the  pulling  of  medical  records  for  probability  sampling.  The 
sampling  sizes  were  ample  for  a  pilot  study  of  this  type  however  it  cannot  be  stated  that  the 
records  are  folly  representative  of  the  individual  clinics,  IMC  and  FPC.  The  possibility  of 
limited  manpower  and  the  need  to  use  clustered  convenience  sampling  was  acknowledged  in 
the  proposal,  however  the  study  was  felt  to  be  of  sufficient  interest  to  pursue  despite  this 


limitation. 


9 


It  appears  likely  that  a  difference  in  the  pneumococcal  vaccination  documentation 
rates  actually  exists  between  the  IMC  and  the  FPC.  It  should  be  noted  that  many  aspects  of 
prevention  and  health  care  maintenance  documentation  were  present  in  the  medical  records 
of  both  clinics.  Both  clinics  made  use  of  the  medical  record  jacket,  problem  lists  of  various 
types,  vaccine  sheets  and  stickers  to  document  pneumococcal  vaccination  in  the  medical 
records.  The  biggest  difference  that  I  noted  was  that  the  IMC  had  an  overprint  of  prevention 
and  health  care  maintenance  items  that  virtually  accompanied  every  patient  visit.  This 
probably  served  as  a  reminder  for  the  provider  to  address  these  issues  at  every  visit.  This  may 
have  helped  the  providers  use  this  as  a  memory  jogger  for  mundane  items  that  they  needed  to 
address  but  were  not  the  principle  reason  for  the  visit.  The  overprint  was  used  liberally  by  the 
providers  in  the  IMC  and  proved  a  useful  reference  in  looking  for  pneumococcal  vaccine 
documentation.  The  IMC  patients  were  older,  see  Tables  2  and  3.  This  may  have  helped 
account  for,  at  least  in  part,  why  they  had  better  documentation.  The  older  patients  had  more 
visits  in  which  to  get  the  pneumococcal  vaccination  documented.  FPC  had  87%  of  their 
patients  between  the  ages  of  65  to  69  whereas;  IMC  had  33.2  %  of  their  patients  in  this  age 
range. 

Philosophically,  both  internal  medicine  and  family  practice  are  primary  care 
specialties  rooted  in  prevention.  WHMC  has  a  residency  program  in  internal  medicine  and 
residents  were  seeing  patients  in  the  internal  medicine  clinic  that  was  surveyed.  The  presence 
of  residents  may  make  all  providers  in  the  clinic  more  attentive  to  documentation  because  of 
the  closer  scrutiny  that  medical  records  get  in  a  residency  program.  It  is  noteworthy  that  the 
training  program  is  considered  one  of  the  best  in  the  country.  The  FPC  does  not  have  any 
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residents  but  it  does  have  mid-level  providers,  physician  assistants  and  nurse  practitioners,  as 
well  as  physicians  seeing  patients.  Though  not  specifically  studied  in  this  survey,  the  mid¬ 
level  providers  and  family  physicians  did  not  appear  to  have  notable  differences  in  terms  of 
prevention  documentation. 

The  large  difference  between  the  documentation  found  in  these  clinics,  92.9%  and 
69%,  and  that  reported  in  the  national  surveys,  34%  to  54.9%,  and  claims  data,  31%,  could 
be  due  to  selection  bias  in  terms  of  the  clinics  chosen  and  charts  selected.2^  Though  this  was 
not  intentional  it  could  very  well  have  been  present.  Initially,  the  hope  was  to  include  two  or 
three  different  facilities  in  the  survey.  However  difficulties  in  getting  cooperation  and 
gaining  access  to  medical  records  in  multiple  facilities  proved  too  much  to  overcome  and  the 
survey  ultimately  evolved  into  a  single  facility  with  two  clinics.  The  survey  facility  did  not 
have  the  manpower  to  allow  for  pulling  of  medical  records  for  a  probability  sample  and  a 
cluster  convenience  sample  was  settled  on.  Therefore,  the  clinic  chosen  and  the  medical 
records  used  in  the  survey  may  have  been  biased.  In  addition,  the  sample  size  was  adequate 
for  a  pilot  study  however  it  was  not  of  the  scope  used  in  national  surveys  and  therefore  may 
have  been  subject  to  sampling  error. 

In  conclusion,  this  survey  though  limited  by  some  factors  did  demonstrate  a 
difference  in  documentation  in  two  different  clinic  types.  Furthermore,  it  was  able  to 
illustrate  a  difference  in  self-reported  national  survey  data  on  pneumococcal  vaccination  and 
that,  which  was  documented  in  the  medical  records  of  this  prevalence  survey.  There  still 
appears  to  be  confusion  among  the  general  population  regarding  what  is  the  influenza 
vaccine  versus  the  pneumococcal  vaccine.  Awareness  and  knowledge  regarding  influenza 
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seems  higher,  possibly  because  it  is  an  annual  campaign.  Pneumococcal  vaccination  is 
principally  a  one-time  event.  In  the  context  of  self-reports,  it  is  not  unreasonable  that 
people’s  memory  regarding  its  administration  may  not  be  accurate,  particularly  as  the  years 
go  by.  The  burden  therefore  is  on  the  medical  system  to  improve  documentation  and  provide 
more  portable  methods  for  patients  to  keep  track  of  vital  data  such  as  immunizations.  These 
could  range  from  low  technology  approaches  such  as  improved  and  utilized  vaccine  cards  to 
computerized  databases  that  could  be  securely  accessed.  Surely  the  true  pneumococcal 
vaccination  rates  are  somewhere  between  the  self-reported  figures  and  those  that  could  be 
found  in  medical  record  surveys  and  is  less  than  100%.  So  although  the  figures  may  not  be  as 
bleak  as  they  once  were,  the  focus  needs  to  be  on  continuing  to  educate  the  population  and 
providers  on  the  value  of  giving  and  documenting  pneumococcal  vaccination  in  the  medical 
record.  Finally,  in  this  survey  the  use  of  the  overprint  by  IMC  was  very  helpful  in  locating 
documentation  of  pneumococcal  vaccination  in  the  medical  record  and  should  be  considered 
by  other  clinics.  Both  clinics,  IMC  and  FPC,  would  be  served  by  standardizing  the  location 
of  vaccine  documentation  in  the  medical  record.  Furthermore  in  the  medical  records  that 
lacked  pneumococcal  vaccination  documentation,  many  had  up  to  date  documentation  of  the 
influenza  vaccine  and  the  two  could  have  been  co-administered  safely  and  without  degrading 
the  effectiveness  of  either  vaccine.10  Efforts  should  be  made  to  educate  providers  and  patients 
to  take  advantage  of  all  opportunities  to  avail  themselves  to  this  important  vaccine. 

DISCLAIMER 

The  views  expressed  in  this  article  are  those  of  the  author  and  do  not  reflect  the  official 
policy  or  position  of  the  U.S.  Air  Force,  Department  of  Defense,  or  the  U.S.  Government. 
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TABLE  1 


Total  Counts  and  Percent  Vaccinated  Against  S.  Pneumoniae 
Medical  Records  Surveyed  by  Air  Force  Clinic  Type: 
Internal  Medicine  &  Family  Practice 


Clinic 


IM 


FP 


Total 


Yes  249(92.9%)  69(69.0%)  318(86.4%) 


Vaccinated 


No  19(7.1%)  31(31.0%)  50(13.6%) 


Total  268 


100 


368 


Chi-square  =  35.462,  p  <  0.001,  df=  1 
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TABLE  2 


Age  Demographics  of  the  Medical  Records  Surveyed  for  Vaccination  Against 
S.  Pneumoniae  at  an  Air  Force  Internal  Medicine  Clinic  &  Family  Practice  Clinic 


Clinic 


Age 


IM 

FP 

Means 

73.4 

67.0 

S.D. 

6.2 

3.9 

Range 

65-93 

65-85 

N 

268 

100 
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TABLE  3 


Age  Breakdown  by  Five-Year  Intervals  of  the  Medical  Records  Surveyed  for 
Vaccination  Against  S.  Pneumoniae  at  an  Air  Force 
Internal  Medicine  Clinic  &  Family  Practice  Clinic 


Clinic 


IM 


FP 


65-69 

89  (33.2%) 

87  (87.0%) 

70-74 

78  (29.1%) 

6  (6.0%) 

75-79 

52(19.4%) 

4  (4.0%) 

80-84 

34(12.7%) 

1  (1.0%) 

85-89 

12(4.5%) 

2  (2.0%) 

90-94 

3  (1.1%) 

0  (0.0%) 

Total  268  (100%) 


100(100%) 
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TABLE  4 

Gender  Demographics  of  the  Medical  Records  Surveyed  for  Vaccination  Against 
S.  Pneumoniae  at  an  Air  Force  Internal  Medicine  Clinic  &  Family  Practice  Clinic 

Clinic 

IM  FP  Total 

Males  132  (49.3%)  43  (43.0%)  175(47.6%) 

Gender 

Females  136(50.7%)  57(57.0%)  193(52.4%) 


Total 


268 


100 


368 


17 


TABLE  5 


Absolute  and  Relative  Frequencies  of  Grades  Found  in  Records  Surveyed  for 
Vaccination  Against  S.  Pneumoniae  at  an  Air  Force 
Internal  Medicine  Clinic  &  Family  Practice  Clinic 


Clinics 


IM 

FP 

Relative 

Relative 

N  Frequency  % 

N  Frequency  % 

E4-E6 

70 

26.1 

37 

37.0 

E7-E9 

114 

42.5 

42 

42.0 

01-03 

11 

4.1 

0 

0.0 

04-06 

69 

25.8 

20 

20.0 

07-above 

4 

1.5 

1 

1.0 

Total 

268 

100.0 

100 

100.0 
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TABLE  6 

Total  Counts  and  Percents  Vaccinated  Against  S.  Pneumoniae 
Records  Surveyed  in  Internal  Medicine:  Officer  or  Enlisted 
IM 

Officer  Enlisted  Total 

Yes  81(96.4%)  168(91.3%)  249(92.9%) 

Vaccinated 

No  3(3.6%)  16(8.7%)  19(7.1%) 

Total  84  184  268 

Chi-square  =  2.299,  p  =  0.129,  df=  1 

TABLE  7 

Total  Counts  and  Percent  Vaccinated  Against  S.  Pneumoniae 
Records  Surveyed  in  Family  Practice:  Officer  or  Enlisted 
FP 

Officer  Enlisted  Total 

12(57.1%)  57(72.2%)  69(69.0%) 

9  (42.9%)  22  (27.8%)  3 1  (3 1 .0%) 

"21  79  100  " 


Yes 

Vaccinated 

No 

Total 


Chi-square  =  1 .747,  p  =  0. 1 86,  df  =  1 


Percent  vaccinated 


ft- 


100 


92.9 


Figure  1:  Bar  chart  of  Pnuemococcal  Vaccination  in  persons  65 
and  older  from  various  surveys 
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